Time series reconstructing using calibrated reservoir computing

Reservoir computing, a new method of machine learning, has recently been used to predict the state evolution of various chaotic dynamic systems. It has significant advantages in terms of training cost and adjusted parameters; however, the prediction length is limited. For classic reservoir computing, the prediction length can only reach five to six Lyapunov times. Here, we modified the method of reservoir computing by adding feedback, continuous or discrete, to “calibrate” the input of the reservoir and then reconstruct the entire dynamic systems. The reconstruction length appreciably increased and the training length obviously decreased. The reconstructing of dynamical systems is studied in detail under this method. The reconstruction can be significantly improved both in length and accuracy. Additionally, we summarized the effect of different kinds of input feedback. The more it interacts with others in dynamical equations, the better the reconstructions. Nonlinear terms can reveal more information than linear terms once the interaction terms are equal. This method has proven effective via several classical chaotic systems. It can be superior to traditional reservoir computing in reconstruction, provides new hints in computing promotion, and may be used in some real applications.

1 Reconstruction results of the coupled Lorenz system in different states 1.1 Reconstruction results of the 6-dimensional system We extend the application system to the 6-dimensional coupled Lorenz system, which is composed of a one-dimensional lattice with two chaotic Lorenz systems. The specific calculation equations are as follows: where σ = 0.01. Setting the initial values as (10.03, 14.69, 22.11, 10.04, 14.68, 22.10), the iteration step is set as 0.002, the total iteration step is 10 5 , and then a 6 × 10 5 data set is obtained. Using the 6-dimensional data obtained from the numerical calculation of the system, we also reconstructed the data by using the "calibrated" reservoir computing. In the 6-dimensional coupled Lorenz system, we use x1 and x2 or y1 and y2 or z1 and z2 in the system as one variable to reconstruct another variable. For example, in the model yz − x, y1, y2, z1, z2 are used as the measured variables, and the reconstructed values are x1, x2. The results of different modes in the coupled Lorenz system are shown in Fig. S1 and Table SI. We demonstrate that using our proposed "calibrated" reservoir computing can make an accurate reconstruction of the coupled Lorenz system in different states. However, it is obvious that the reconstruction error of x − yz mode is much larger than that in other modes. It may be caused by the coupling of variable x, and the training set is larger when variables are coupled than without coupling, such as xy − z and xz − y modes. With the increase of spatial coupling part and dimension, maintaining the same reconstruction length, a larger training set needs to be added, otherwise the reconstruction length will become shorter. Furthermore, we consider the case where two subsystem variables are used independently, i.e., x1y1z1 (or x2y2z2) of one system is used to reconstruct x2y2z2 (or x1y1z1) of the other. The results of different modes in the coupled Lorenz system are shown in Fig. S2 and Table SII. Its findings demonstrate that regardless of which side of the variable is used to reconstruct the other side, the reconstruction effect is essentially the same. This result could be explained by the fact that the parameter settings of the two subsystems are consistent, so there is no difference in the reconstruction effect. It should be noted that RMSE in all tables is the average of all reconstructed variables.

Reconstruction results of the 15-dimensional system
In this section, we extend the "calibrated" reservoir computing in the 15-dimensional Lorenz system to study the relationship between the performance of reconstruction and the number of the measured variables. Specific parameter settings are as follows: where j ̸ = i, i, j = 1, 2, 3, 4, 5, σ = 0.01. Using the obtained data, we have conducted a detailed study on the reconstruction performance of the "calibrated" reservoir on high-dimensional systems. The results are shown in Fig. S3 and Table SIII. As the dimension of the system grows, so does the calculation difficulty. We choose the yz − x mode with the best reconstruction effect according to the calculation results, x, y, z respectively represent all xi, yi, zi in each subsystem. Then, we try to reduce the number of the measured variables to make an accurate reconstruction in the yz − x mode, i.e., reduce the number of the measured variables yz. In the yz − x − 1 mode, for example, the measured variables are y1, z1, and the reconstructed variables are x1, x2, x3, x4, x5, y2, y3, y4, y5 and z2, z3, z4, z5. In the yz − x − 2 mode, the measured variables are y1, y2, z1, z2, and the reconstructed variables are x1, x2, x3, x4, x5, y3, y4, y5 and z3, z4, z5. The modes yz − x − 3 and yz − x − 4 are also similar. The specific outcomes are as follows: The system's reconstruction effect appears to weaken as the number of the measured variables decreases. However, whether in a single or coupled system, we find that if the yz variable of the subsystem  is given, the x variable of the subsystem can be reconstructed. This also implies that "calibrated" reservoir computing may be better able to obtain the interaction of various variables within the system, whereas learning the spatial coupling relationship may be more difficult. Finally, after testing each model, we can reach the following conclusions: When the reconstruction length T test is 500, the number of observed variables in the 15-dimensional coupling system must be approximately 53.3%(8/15) in order to achieve accurate reconstruction. The results are shown in Fig. S4 and Table SIV. When T test is 100, the number of the measured variables required to achieve accurate reconstruction is only about 26.7%(4/15). The results are shown in Fig. S5 and Table SV.
In addition, we also constructed the xyz mode, that is, using the xyz variables of several of the five subsystems to reconstruct the xyz variables of the remaining subsystems. For example, in the xyz − 1 mode, the measured variables are x1, y1 and z1, while the remaining variables x2, x3, x4, x5, y2, y3, y4, y5 and z2, z3, z4, z5 are the reconstructed variables. The measured variables in the xyz − 2 mode are x1, x2, y1, y2 and z1, z2, while the remaining variables are x3, x4, x5, y3, y4, y5 and z3, z4, z5. The xyz − 3 and xyz − 4 modes are interchangeable. When the reconstruction length T test is set as 500, the number of the measured variables needs to reach 80%(12/15) to accurately reconstruct the 15-dimensional system. When the reconstruction length T test is set to 100, at least 40%(6/15) measured variables are required for accurate reconstruction. The results are shown in Fig. S6, Fig. S7 and Table SVI, Table SVII. From the above experimental results, we can see that the reconstruction error RMSE of all modes has been significantly reduced with the shortening of T test . It also implies that in a higher-dimensional system, if we want to perform accurate reconstruction for a long time, we can increase T train appropriately. Furthermore, the requirement for accurate reconstruction in xyz mode is far greater than in yz − x mode, indicating that the x variable is unimportant in the reconstruction of this system. All the above experimental results further demonstrate that the "calibrated" reservoir computing can still perform accurate reconstruction in high-dimensional systems.

Reconstruction results of Rössler systems in different states
We demonstrate that using our proposed "calibrated" reservoir computing can make an accurate reconstruction of the classical Rössler 1 system in different states.
where A = 0.15, B = 0.2. Firstly, we obtain the time series of different states by changing the parameter C. (i.e.,Rössler1: C = 3.4; Rössler2: C = 4.4; Rössler3: C = 5.5; Rössler4: C = 5.9.) The time series of one-period, two-period, four-period and eight-period Rössler system are obtained by numerical calculation, and the integration step ∆t = 0.02. During the training phase, the parameter setting of the "calibrated" reservoir computing is η = 1 × 10 −8 , the average degree D = 9.5, the bias constant ξ =0.1, and the reservoir network nodes N = 95.  Similarly, we select the shortest training length as the index of reconstruction effect when the reconstruction length is fixed as 10 4 and the RMSE <0.1. The results of different Rössler systems are shown in Fig. S8 and Table SVIII. Note that, the hyperparameters are adjusted to get the shortest training length in each mode.
We can see that whether the states of Rössler is periodic or chaotic, the best reconstruction effect always is x − yz mode. The results show there is a relationship between the efficiency of reconstruction and the equation structure. The variable x, which appears in both the equationsẏ,ż, performs best at reconstructing the variables y, z in all cases.

Reconstruction results of some classical dynamical systems
In this section, we test the "calibrated" reservoir computing model in more classical dynamical systems, where the parameters are all set as chaotic states. The corresponding equations of these systems are shown below: Shimizu-Morioka(S-M) 2 system: where, A = 0.75, B = 0.45.
Wang-Sun(W-S) 5 system: where, A = 0.2, B = −0.01,C = −0.4. For these systems, the parameter setting of the "calibrated" reservoir computing is η = 1 × 10 −8 , the average degree D = 9.5, the bias constant ξ =0.1, and the number of reservoir network nodes N = 95. The integration time step is ∆t = 0.02. And the reconstruction results are shown in Fig. S9 and Table SIX. All of these results indicate that the more interactions a variable has with other in dynamical equations, the better reconstruction of this variable is in reconstructing other. These systems, however, cannot completely confirm the relationship between the nonlinear terms and the reconstruction.

Reconstruction results of "xy-yz" and "xz-yz" systems
In this section, we build two systems to verify the relation between the efficiency of reconstruction and nonlinear terms for the other two situations (when the system contains xy, yz and xz, yz), the math equations are as follows: where,A = 14.1, B = 5.0,C = −18.8, D = −21.5, E = 2.0.
where, A = 3.17, B = 3.93,C = 8.04, D = 12. During the training phase, the parameter setting of the "calibrated" reservoir computing is η = 1 × 10 −8 , the average degree D = 9.5, the bias constant ξ =0.1, and the number of reservoir network nodes N = 95. The integration time step is ∆t = 0.02. The reconstruction results are shown in Fig. S10 and Table SX, we can see that the reconstruction of the xz − y mode is better than the other modes (yz − x, xy − z) in the "xy-yz" system. It is conformed to our finding: the variable y is the repeated one, so it can be reconstructed to the best. Likewise, the best reconstruction in the "xz-yz" system is the xy − z mode, due to the variable z being the repeated one in the nonlinear terms(xy, yz). All these results indicate that the efficiency of reconstruction of the system can play a certain role in revealing the dynamic features. xy-yz xz-yz t t Figure S10. The time diagrams and the difference between reconstructed values and actual values of "xy-yz" and "xz-yz" systems. (a), (b) correspond to the best mode of "xy-yz" and "xz-yz" systems respectively.